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Abstract — The capacity region of the A'^-sender Gaussian mul- 
tiple access channel with feedback is not known in general. This 
paper studies the class of linear-feedback codes that includes 
(nonlinear) nonfeedback codes at one extreme and the linear- 
feedback codes by Schalkwijk and Kailath, Ozarow, and Kramer 
at the other extreme. The linear-feedback sum-capacity Cl{N, P) 
under symmetric power constraints P is characterized, the 
maximum sum-rate achieved by linear-feedback codes when each 
sender has the equal block power constraint P. In particular, it 
is shown that Kramer's code achieves this linear-feedback sum- 
capacity. The proof involves the dependence balance condition 
introduced by Hekstra and Willems and extended by Kramer and 
Gastpar, and the analysis of the resulting nonconvex optimization 
problem via a Lagrange dual formulation. Finally, an observation 
is presented based on the properties of the conditional maximal 
correlation — an extension of the Hirschfeld-Gebelein-Renyi max- 
imal correlation — which reinforces the conjecture that Kramer's 
code achieves not only the linear-feedback sum-capacity, but 
also the sum-capacity itself (the maximum sum-rate achieved 
by arbitrary feedback codes). 

Index Terms — Feedback, Gaussian multiple access channel, 
Kramer's code, linear-feedback codes, maximal correlation, sum- 
capacity. 



I. Introduction 

Feedback from the receivers to the senders can improve the 
performance of the communication systems in various ways. 
For example, as first shown by Gaarder and Wolf |1J, feedback 
can enlarge the capacity region of memoryless multiple access 
channels by enabling the distributed senders to cooperate via 
coherent transmissions. 

In this paper, we study the sum-capacity of the addi- 
tive white Gaussian noise multiple access channel (Gaussian 
multiple access channel in short) with feedback depicted in 
Figure [T] For N — 2 senders, Ozarow f2l established the ca- 
pacity region which — unlike for the point-to-point channel — is 
strictly larger than the one without feedback. The capacity- 
achieving code proposed by Ozarow is an extension of the 
SchaUcwijk-Kailath code ||3l, lU for Gaussian point-to-point 
channels. 

For iV > 3, the capacity region is not known in general. 
Thomas [51 proved that feedback can at most double the 
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sum capacity, and later Ordentlich f6\ showed that the same 
bound holds for the entire capacity region even when the 
noise sequence is not white (cf. Pombra and Cover Q). More 
recently, Kramer |8| extended Ozarow's linear-feedback code 
to iV > 3 senders, and proved that this code achieves the sum- 
capacity under symmetric block power constraints P on all the 
senders, when the power P is above a certain threshold (see (01 
in Section |ll]i that depends on the number of senders N. 

In this paper, we focus on the class of linear-feedback codes, 
where the feedback signals are incorporated linearly into the 
transmit signals (see Definition [T] in Section This class of 
codes includes the hnear-feedback codes by Schalkwijk and 
Kailath f3l, Ozarow fT\, and Kramer |8| as well as arbitrary 
(nonlinear) nonfeedback codes. 

We characterize the linear-feedback sum-capacity Ci^ (N, P) 
under symmetric block power constraints P, which is the 
maximum sum-rate achieved by linear-feedback codes under 
equal block power constraints P at all the senders. Our main 
contribution is the proof of the converse. We first prove an 
upper bound on C\,{N,P), which is a multiletter optimiza- 
tion problem over Gaussian distributions satisfying a certain 
functional relationship (cf. Cover and Pombra |f9l|). Next, we 
relax the functional relationship by considering a dependence 
balance condition, introduced by Hekstra and Willems lITOI 
and extended by Kramer and Gastpar and derive an 

optimization problem over the set of positive semidefinite (co- 
variance) matrices. Lastly, we carefully analyze this nonconvex 
optimization problem via a Lagrange dual formulation |12|. 

The linear-feedback sum-capacity Cl (iV, P) is achieved 
by Kramer's linear-feedback code. Hence, this rather sim- 
ple code, which iteratively refines the receiver's knowledge 
about the messages, is sum-rate optimal among the class of 
linear-feedback codes. For completeness, we briefly describe 
Kramer's linear-feedback code and analyze it via properties of 
discrete algebraic Riccati recursions (cf. Wu et al. 113|). This 
analysis differs from the original approaches by Ozarow |]2| 
and Kramer 18J. 

The complete characterization of the sum-capacity C{N, P) 
under symmetric block power constraints P, i.e., the maximum 
sum-rate achieved by arbitrary feedback codes, still remains 
open. However, it has been commonly believed (cf. ifTTI .l fTsl ) 
that linear-feedback codes achieve the sum-capacity, i.e., 
C{N,P) = Cl(A^, P). We offer an observation that further 
supports this conjecture. By introducing and analyzing the 
properties of conditional maximal correlation, which is an 
extension of the Hirschfeld-Gebelein-Renyi maximal corre- 
lation IIT4I to the case where an additional common random 
variable is shared, we show in Section |V] that the linear- 
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Fig. 1. A'^-sender Gaussian multiple access channel. 



feedback codes are greedy optimal for a multiletter optimiza- 
tion problem that upper bounds C{N, P). 

The rest of the paper is organized as follows. In Section HIl 
we formally state the problem and present our main result. 
Section Hill provides the proof of the converse and Section HVl 
gives an alternative proof of achievability via Kramer's linear- 
feedback code. Section W\ concludes the paper with a discus- 
sion on potential extensions of the main ideas to nonequal 
power constraints and arbitrary feedback codes, and with a 
proof that linear-feedback codes are greedy optimal for a 
multiletter optimization problem that upper bounds C{N,P). 

We closely follow the notation in |15|. In particular, a ran- 
dom variable is denoted by an upper case letter (e.g., X, Y, Z) 
and its realization is denoted by a lower case letter (e.g., 
X, y, z). The shorthand notation X" is used to denote the tuple 
(or the column vector) of random variables {Xi,...,Xn), 
and a;" is used to denote their realizations. A random column 
vector and its realization are denoted by boldface letters (e.g. 
X and x) as well. Uppercase letters (e.g.. A, B, C) also 
denote deterministic matrices, which can be distinguished from 
random variables based on the context. The (i, j) element of a 
matrix A is denoted by Aij . The conjugate transpose of a real 
or complex matrix A is denoted by A' and the determinant 
of A is denoted by \A\. For the crosscovariance matrix of 
two random vectors X and Y, we use the shorthand notation 
Kx.Y ■■= E(XY') - E(X) E(Y') and for the covariance matrix 
of a random vector X we use Kx '■= ^^^xx- Calligraphic 
letters (e.g., A,B,C) denote discrete sets. Let {Xi, . . . ,Xn) 
be a tuple of N random variables and A S {!,..., N}. 
The subtuple of random variables with indices from A is 
denoted by X{A) :— {Xj : j e A). For every positive real 
number m, the short-hand notation [1 : 2™] is used to denote 
the set of integers {1, . . . , 2^"'^ }. 

II. Problem Setup and the Main Result 

Consider the communication problem over a Gaussian mul- 
tiple access channel with feedback depicted in Figure [T] Each 
sender j G {1,...,N} wishes to transmit a message Mj 
reliably to the common receiver. At each time i = 1, . . . ,n, 
the output of the channel is 



N 



(1) 



where {Zi\ is a discrete-time zero-mean white Gaussian 
noise process with unit average power, i.e., E{Zf) = 1, 
and is independent of {Mi, . . . , Mn). We assume that the 
output symbols are causally fed back to each sender, and 
that the transmitted symbol Xji from sender j at time i can 
thus depend on both the previous channel output sequence 
Y'-^ := (Yi,Y2,...,Yi^i) and the message Mj. 



We define a (2 



nRi 



,2"-«",n) 
,...,Mn- 



feedback code as 
where Mj := [1 : 2"-^^] 



1) message sets A^i 
for j = l,...,7V; 

2) a set of N encoders, where encoder j e {1,...,A^} 
assigns a symbol Xji{mj, to its message rrij £ Aij 
and the past channel output sequence y^^^ E W^'^ for 
I € {1, ... , n}; and 

3) a decoder that assigns message estimates ?fij € [1 : 2"^^ ], 
j e {1, . . . , N}, to each received sequence y". 

We assume throughout that M{S) := (Mi,..., Mat) is 
uniformly distributed over [1 : 2"^i] x • • • x [1 : 2"^"]. The 
probability of error is defined as 

pj") P{M{S) ^ M{S)}. 



A rate tuple J^Ri , . . 
rate R = J2j=i^j 



■ , Rn) and its corresponding sum- 
are said to be achievable under the 



power constraints (Pi, . . . ,Pn) if there exists a sequence of 
(2"^\ . . . , 2"^", n) feedback codes such that the expected 
block power constraints 



n 

-^E(x2(M„y-i))<p„ 



j = l,...,Ar 



are satisfied and lim„_j.oo Pe"'^ = 0. The supremum over 
all achievable sum-rates is referred to as the sum-capacity. 
In most of the paper, we will be interested in the case of 
symmetric power constraints Pi = P2 = • • • = Pn — P- In 
this case we denote the sum-capacity by C{N,P). 

Our focus will be on the special class of linear-feedback 
codes defined as follows. 

Definition 1: A (2"^% . . . , 2"-^", n) feedback code is said 
to be a linear-feedback code if the encoder Xji{mj, y*^^) has 
the form 

Xj^ = Lj,{9j{mj),y'-^), 
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where 

1) the (potentially nonlinear) nonfeedback mapping 6j is 
independent of i and maps the message rrij to a k- 
dimensional real vector (message point) 9j for some 

e {1, . . . , n}; and 

2) the linear-feedback mapping Lji maps the message point 
6j{mj) and the past feedback output sequence y'^^ to 
the channel input symbol Xji. 

The class of linear-feedback codes includes as special cases 
the feedback codes by Schalkwijk and Kailath |3 1, Ozarow [2J, 
and Kramer |8|, and all nonfeedback codes. To recover the 
codes by Schalkwijk and Kailath [SJ and Ozarow liZ| it suffices 
to choose fc = 1; for Kramer's code HI we need k = 2; and 
to recover all nonfeedback codes we have to choose k — n 
and each message point 6j equal to the codeword sent by 
encoder j. 

The linear-feedback sum-capacity is defined as the maxi- 
mum achievable sum-rate using only linear-feedback codes. 
Under symmetric block power constraints Pi = ■ ■ ■ = Pn = 
P, we denote the linear- feedback sum-capacity by Cl(-/V, P). 

We are ready to state the main result of this paper. 

Theorem 1: For the Gaussian multiple access channel with 
symmetric block power constraints P, the linear-feedback 
sum-capacity is 

Ci^iN, F) = i log(l + NPcf^iN, P)) (2) 

where <j>{N, P) is the unique solution to 

{l + NPcbf-' = {l + P^{N-<j))f (3) 

in the interval [1,-/V]. 

The proof of Theorem [T] has several parts. The converse is 
proved in Section The proof of achievability follows by 
m Theorem 2] and can be proved based on Kramer's linear- 
feedback code [8J. For completeness, we present a simple 
description and analysis of Kramer's code in Section |IV] 
Finally, the property that (O has a unique solution in [1,A^] 
is proved in Appendix |A] 

Remark 1: Kramer showed [8] that when the power con- 
straint P exceeds the threshold Pc{N), which is the unique 
positive solution to 

{l + N^P/2f-^ = {l+N^P/4)^, (4) 

then the sum-capacity C {N, P) is given by the right-hand 
side of (|2]i. Thus, for this case Theorem [T] follows directly 
from Kramer's more general result. Consequently, when P > 
Pc{N), then the linear-feedback sum-capacity coincides with 
the sum-capacity, i.e., Ci,{N, P) = C{N, P). It is not known 
whether this equality holds for all powers P; see also our 
discussion in Section IV-BI 

Remark 2: Since (t){N, P) E [i-, N], we can define a param- 
eter p G [0, 1] so that (j){N, P) = 1 + (N - l)p. Intuitively, p 
measures the correlation between the transmitted signals. For 
example, when N = 2, the corresponding p coincides with 
the optimal correlation coefficient p* in 12J. Thus, (j)(N,P) G 
[1,N] captures the amount of cooperation (coherent power 
gain) that can be established among the senders using linear- 
feedback codes, where (p — 1 corresponds to no cooperation 



and (j) — N corresponds to full cooperation. For a fixed N > 2, 
(piN, P) is strictly increasing (see Appendix |A]i; thus, more 
power allows for more cooperation. Moreover, (^{N, P) — 7> 1 
as P — ^ and (j>{N, P) — as P oo, which is seen as 
follows. We rewrite identity (O as 

/ P02 

and notice that the left-hand side (LHS) of (|5]l can be written 

as 1 + P(lP{N - 1) + o(P), where o(P) tends to faster than 
P. Thus, the LHS of ^ can equal its right-hand side (RHS) 
only if cjP{N — 1) — (j){N —(f) — > as P — > 0, or equivalently, 
<j){N, P) ^ 1 as P ^ 0. On the other hand, as P ^ oo, the 
LHS tends to a constant while the RHS tends to infinity unless 
N — (f> tends to 0. Thus, by contradiction, 4>{N, P) ^ N as 
P — > oo. 

By the above observation, we have the following two 
corollaries to Theorem [T] for the low and high signal-to-noise 
ratio (SNR) regimes. 

Corollary 1: In the low SNR regime, almost no cooperation 
is possible and the linear-feedback sum-capacity approaches 
the sum-capacity without feedback: 

j^im [ci^{N, P) - i log(l + NP)^ = 0. 

Corollary 2: In the high SNR regime, the linear-feedback 
sum-capacity approaches the sum-capacity with full coopera- 
tion where all the transmitted signals are coherently aligned 
with combined SNR equal to A^^P: 

lim (ci^{N, P) - i log(l + N^P)] = 0. 

III. Proof of the Converse 

In this section we show that under the symmetric 
block power constraints P, the linear-feedback sum-capacity 
Cl ( A^, P) is upper bounded as 

P)<\ log(l + NP<\>(N, P)) (6) 

where <p(N,P) £ [l,Af] is defined in Q. 

The proof involves five steps. First, we derive an upper 
bound on the linear-feedback sum-capacity based on Fano's 
inequality and the maximum entropy property of Gaussian 
distributions (see Lemma[T]l. Second, we relax the problem by 
replacing the functional structure in the optimizing Gaussian 
input distributions dS) with a dependence balance condition 
ifTOl . ITIi . and we rewrite the resulting nonconvex optimiza- 
tion problem as one over positive semidefinite matrices (see 
Lemma |2|i. Third, we consider the Lagrange dual function 
J(A, 7), which yields an upper bound on C\{N, P) for every 
A, 7 > (see Lemma |3]l. Fourth, by exploiting the convexity 
and symmetry of the problem, we simplify the upper bound 
J(A, 7) into an unconstrained optimization problem (which is 
still nonconvex) that involves only two optimization variables 
(see Lemma IHi. Fifth and last, using brute-force calculus and 
strong duality, we show that there exist A*, 7* > such that 
the corresponding upper bound J(A*,7*) coincides with the 
right hand side of (|6]l (see Lemma |5]l. 
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The details are as follows. 

Lemma 1: The linear-feedback sum-capacity C-l{N,P) is 
upper bounded as 



CL(iV,F) < lim sup C["'(P), 



wher^H 



n 

Cf^ (P) max - V . . . , Xm;Y,\Y'~') 



(7) 



i=l 



and the maximum is over all inputs Xji of the form 



X,, = L,,(y,„y^-i), z = l,...,n, j-l,...,iV, (8) 

such that the function Lji is linear, the vector Vj G R" ^ 
N(0, AV ) is Gaussian, independent of the noise vector Z" 
and the tuple (Vj' : j' ^ j), and the power constraint 
X;r=i H^jt) < nP is satisfied. 

Proof: By Fano's inequality lfT6l . 

AT 

i/(A/(5)|F") < 1 + nPi") ^ i?, ne„ 

for some e„ that tends to zero along with Pe as n — >^ oo. 
Thus, for any achievable rate tuple {Ri , . . . , Rn), the sum-rate 
R can be upper bounded as follows: 

N 



iR ^ n Rj 
i=i 
= i?(Af(5)) 
</(M(5);r")+ne„ 
< 7(0(5); 

<^/(x,(5);r.|r-i) 



(9) 
(10) 

(11) 



where ( fTOl i and (fTTT i follow by the data processing inequality 
and the memoryless property of the channel, respectively. 
Therefore, the linear-feedback sum-capacity is upper bounded 
as 



1 



n 



Cl(A^,P) <limsup-max^/(X,(5);r,|r*"^), (12) 

z— 1 



where the maximum is over all input distributions induced 
by a linear-feedback code satisfying the symmetric power 
constraints P, i.e., over all choices of independent random 
vectors 0i,...,0jv and linear functions Lji such that the 
inputs Xji — Lji{@j,Y^^^) satisfy the power constraints 
Er=i HX],) < Now let 



V, ^N(0,i^©J, j = l,. 



,iV, 



be a Gaussian random vector with the same covariance matrix 
as @j, independent of (Vj/ : / 7^ j). Using the same linear 
functions Lji as in the given code, define 

1,, -i,,(V„f^-i), (13) 

'For simplicity of notation we do not include the parameter A'^ explicitly 
in most functions that we define in this section, e.g., C^"'(P). 



where % is the channel output of a Gaussian MAC corre- 
sponding to the input tuple Xi{S). It is not hard to see that 
{Xi{S),Y'^) is jointly Gaussian with zero mean and of the 
same covariance matrix as {Xi{S),Y'^). Therefore, by the 
conditional maximum entropy theorem [|5J Lemma 1 ] we have 



L{X,{S);Y,\Y'-') < 



(14) 



Combining (fT2] l and ( fT4] l and appropriately defining Yj in ^ 
from Vj in (fT3T l completes the proof of Lemma [T] ■ 

We define the following functions on N-by-N covariance 
matrices K: 



/i(A') = ilog(l + ^if,y 



(15a) 



N 



2{N 



1 + E ^rr' - 

3',j" 



33' ) 



K 



33 



(15b) 



It can be readily checked that both functions are concave in K 
(see Appendix iBl). 

Lemma 2: The linear-feedback sum-capacity Cl (iV, P) is 
upper bounded as 

1 " 

Cl(A^,P) < limsup max -S^ MKi), (16) 
- Ki,...,Kn n ^-^ 



i=l 



where the maximum is over A^-by-A^ covariance matrices 
{Ki h 0}^^i such that 



n 

E/l(^^)"/2(i^.) <0. 



(17) 



(18) 



Proof: Since Xji is defined by the (causal) functional 
relationship in (O, by ifTOl . ifTTl Theorem 1] we have the 
dependence balance condition 

n 

Y^I{X,{S)-Y,\Y^-^) 

^ 71 N 

4=1 i=i 

Furthermore, recall that (X"(5), F") is jointly Gaussian. 
Therefore, for every i e {1, . . . , n}, conditioned on Yi-i — 
y^^^, the input (column) vector :— [Xu, . . . , XNi) is zero- 
mean Gaussian with covariance matrix 

irrespective of y*^^. Now consider 

I{X,{S);Y,\Y'-^) - h{Y,\Y'-') - h{Z,) 
= ilog(Var(r,|y^-i)) 

= \\og(l + Y,{K.)33') 

3,3' 

= fi{K,). (20) 
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Also consider 



Var {Y,\X,„Y'-^) = I + iK^)rr' - 



IJ33 



which implies that 

AT 



N 



L_J2liX.iS\{j});nY^-\X,,) - (21) 



Hence, condition ( fT9] l reduces to ( fTSl l. Rewriting Q in terms 
of covariance matrices Ki via ( |20l i and relaxing the functional 
relationship dHJ by the dependence balance condition (fTSl l 
completes the proof of Lemma |2] ■ 

Remark 3: Although both functions fi{K) and f2{K) are 
concave, their difference [2 [K] — fi [K] is neither concave nor 
convex. Hence, the optimization problem in ( fT6l l is nonconvex. 

Lemma 3: Let fi{K) and J2{K) be defined as in ( I15ab 
and ( llSbl i. Then for every A, 7 > 0, 

< J(A,7), (22) 

where 



J(A,7):=max {I - -1) h{K) + -1 h{K) 



N 



(23) 



Proof: By the standard Lagrange duality fT2\, for any 
Ai, . . . , Aat, 7 > 0, the maximum in (fTST i is upper bounded as 



max 
Ki,...,Kk n 



< 



K 



1 " 
rj — ^ 



1 " 

max — > 

i,...,/fjv n ^ 



N 



where the maximum is over Ki , . . . , h (without any 
other constraints). Here, Ai,...,Ajv > are the Lagrange 
multipliers corresponding to the power constraints ([TtT i and 
7 > is the Lagrange multiplier corresponding to the 
dependence balance constraint (fTsT l. Finally, we choose Ai = 
• • • = \n = A, which yields 



1 " 



ifi,...,is'jv n 



< max 
A'l,...,^™ n 



1 " 

— >^ max 

n Ki 



1 " 

-E 



hiK,)+^{f2{K^)-h{K,)) 



N 



N 



= J(A,7), 
and completes the proof of Lemma [3] 



Lemma 4: For every A, 7 > 0, 



J(A,7)=max max 17(7, x, 0) + AiV(P — x), 

a;>0 0<(p<N 



where 



and 



g{'j,x,(j)) := (1 - 7)Ci(a;,(/)) +7C2(a;, 
1 



Ci(x,0) 



log(l + A^x? 



■log(l + {N -<t))x4 



(24) 
(25) 

(26a) 
(26b) 



2(A^- 1, 

Proof: Suppose that a covariance matrix K attains the 
maximum in ( |23] |. For each permutation tt on {1, . . . , A^}, let 
tt{K) be the covariance matrix obtained by permuting the rows 
and columns of K according to tt, i.e., {TT{K))jji — KT^i^j^^i^j:-^ 
for J, / e {l,...,Ar}. Let 



TT 

be the arithmetic average of K over all A^! permutations. 
Clearly, K is positive semidefinite and of the form 

f 1 p ■■■ p\ 
p 1 ■■■ p 



K = x- 



(27) 



\p p ■■■ 1 / 

for some a: > and — 1/(A^ — 1) < p < 1- (The conditions 
on X and p assure that K is positive semidefinite.) We now 
show that also K attains the maximum in ( |23] ). First, notice 
that the function /i (A') depends on the matrix K only via the 
sum of its entries and hence 

/i(if) = /i(7r(X)) = /i(/0. 

Similarly, 

N N N 

Also, by symmetry we have f2(K) = /2(7r(Ar)). Hence, 
by the concavity of f2{K) (see Appendix iBl) and Jensen's 
inequality, f2{K) < f2{K). Therefore, 

N 



(1 - 7)/i(if) + lf2{K) + AE(^ - Kj, 



N 



< (1 - 7)/i(i?) + 7/2(i?) + A E(^ - K,,) 

i=i 

and the maximum of ( [23T l is also attained by K. Finally, 
defining (f) :— 1 + {N — l)p e [0,N] and simpUfying dlSal ) 
and dlSbt yields 

which completes the proof of Lemma |4] ■ 



Remark 4: The symmetric K in (|27| | was also considered in 
in, (8] to evaluate the cutset upper bound, which corresponds 
to taking 7 < 1. 
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Lemma 5: There exist A*, 7* > such that 

j{x*,j*)<ci{p,m,p)) 

= ^logil + NP<j>{N,P)), 

where (j>{N, P) is defined in 

Proof: Consider the optimization problem over (a;, (f), 
which defines J(A,7) in ( l24l i. Note that g{'y,x,(j)) given 
by (IZST i is neither concave or convex in (a:, cj)) for 7 > 1. 
However, 5(7, x, (p) is concave in > for fixed x, 7 > as 
shown in Appendix ICl 

Let (p* — 0*(7,x) be the unique nonnegative solution to 



0, 



or equivalently to 

(l-7)(iV-l) ^ 7(20* - N) 

l + Nxcp* 1 + x(j)*{N ~ <j)*)' ' ' 

(That such a unique solution exists is easily verified consider- 
ing the equivalent quadratic equation; see dTOl l in Appendix ID]) 
Then, by the concavity of 3(7, x, 0) in ip for fixed 7 and x, 

J(A,7)=max max g(^,x,4>) + \N{P — x) 

x>0 0<4><N 

<m&xg{'y,x,(f>*{j,x)) + XN{P - x) (29) 

a:>0 

for any 7 > 0. (The inequality follows because (j>*{'y, x) might 
be larger than N.) 

Now let g*{j,x) = 5(7, x, (/)*(7, x)). Then, g*{'~f,x) is 
nondecreasing and concave in x for fixed 7 as shown in 
Appendix ID] Thus, 

min J(A, 7) < min max g* (7, x) + XN{P — x) 

A>0 A>0 X 

= max.g*(7,x) 

X<P 

^9*{l.P). (30) 

where the first equality follows by Slater's condition |fT2l and 
strong duality, and the last equality follows by the monotonic- 
ity of g*{'-f,x) in x. Alternatively, the equality in (|30] | can 
be viewed as the complementary slackness condition lfT2l . 
Indeed, since 5* (7, x) is not bounded from above, the optimal 
Lagrange multiplier A* > must be positive. Therefore, the 
corresponding constraint a; < P is active at the optimum, i.e., 
X* ^ P. 

Finally, we choose 7 = 7*, where 
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1 



{N - 24){N, P))(l + NP(I){N, P)) 



[N - 1)(1 + P0(7V, P){N - 0(7V, P))) , 

which assures that (/)*(7*,P) coincides with (l){N,P) (see 
(|28]l). Since 7* is nonnegative by ( |57] i in Appendix |A] and 
thus is a valid choice, 

/(7*,P)=g(7*,P,0(iV,P)) 

= (l-7*)Ci(P,0(iV,P))+7*C2(P,0(iV,P)) 
= Ci(P,0(7V,P)), 

which, combined with (|30] l, concludes the proof of Lemma |5] 
and of the converse. ■ 



IV. ACHIEVABILITY VIA KRAMER'S CODE 

We present (a slightly modified version of) Kramer's linear- 
feedback code and analyze it based on the properties of 
discrete algebraic Riccati equations (DARE). In particular, we 
establish the following: 

Theorem 2: Suppose that . . . , > 1 are real numbers 
and wi,...,cjjv are distinct complex numbers on the unit 
circle. Let A = diag(/3iwi, . . . , Pni^n) be a diagonal matrix, 
1 = (1,...,1) be the all-one column vector, and K* be 
the unique positive-definite solution to the discrete algebraic 
Riccati equation (DARE) 

K = AKA' - {AK1){1 + V Kl)-^ {AKiy . (31) 

Then, a rate tuple (i?i, . . . ,Rn) is achievable under power 
constraints (Pi , . . . , Pat), provided that Rj < log (3j and Pj > 

Achievability of Theorem [T] will be proved in Subsec- 
tion IIV-CI as a corollary to Theorem |2] 

A. Kramer's Linear-Feedback Code 

Following (HI, we represent a pair of consecutive uses of 
the given real Gaussian MAC as a single use of a complex 
Gaussian MAC. We represent the message point of sender j by 
the complex scalar Qj (corresponding to fc = 2 in the original 
real channel) and let := (9i,...,8Ar) be the (column) 
vector of message points. 

The coding scheme has the following parameters: real 
coefficients /3i,...,/3jv > 1 and distinct complex numbers 
wi, . . . , cijjv on the unit circle. 

Nonfeedback mappings: For j = 1, . . . , A^, we divide the 
square with corners at {±.1±\/^A} on the complex plane into 
22nRj gq^^J subsquares. We then assign a different message 
rrij e [1 : 2^"^^] to each subsquare and denote the complex 
number in the center of the subsquare by Oj {raj). The message 
point Qj of sender j is then 8j = 9j{Mj). 

Linear-feedback mappings: Let :~ (Xn, . . . , X^^i) 
denote the (column) vector of channel inputs at time i. We 
use the linear-feedback mappings 



Xi =0, 

X, = A-(x,_i-x,_i(r,_i)), i>i 



where 



(32) 
(33) 



A = diag (/Siwi, ^2^2, • . • , /Sjv^at) 
is a diagonal matrix with Ajj = PjUtj and 

is the linear minimum mean squared error (MMSE) estimate 
of Xj_i given Y^^i. 

Decoding: Upon receiving F", the decoder forms a message 
estimate vector 

n-l 

0:=(ei,...,eAr) = ^A-% (34) 

i=0 

and for each j = 1, . . . , chooses A4j such that 6j{Mj) is 
the center point of the subsquare containing Qj. 
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B. Analysis of the Probability of Error 

Our analysis is based on the following auxiliary lemma. We 
use the short-hand notation Ki — Kx. . 
Lemma 6: 

lim Kn = K* (35) 

n— *-oo 

where K* is the unique positive-definite solution to the 
DARE dlB- 

Proof: We rewrite the channel outputs in ^ as 



From ( [32] l we have 



(37) 



where ^i^x -x ^ ^y^i ^ ^^iYi Ky^ K^^.y. is the error 
covariance matrix of the linear MMSE estimate of given Yi. 
Combining (|36] l and d37T i we obtain the Riccati recursion IfTTl 

K,+i = AK,A' - iAK,l)il + l'K,l)-\AKaY (38) 

for i = — 1. Since A has no unit-circle eigenvalue 

and the pair (A, 1) is detectablejl we use Lemma 2.5 in iT9l 
to conclude (l35l l. ■ 
We now prove that Kramer's code achieves any rate tuple 
(i?i, . . . , Rn) such that 



Rj<\ogPj, j^l,...,N. 



(39) 



Define the difference vector D„ := — 0„. Since the 
minimum distance between message points is A = 2 • 2~"^j , 
by the union of events bound and the Chebyshev inequality, 
the probability of error of Kramer's code is upper bounded as 

Fi")<p(|J{|D„(j)|>A/2} 



J 

N 

<^22„fl, E(|D„(j)n. 
i=i 



(40) 



Rewriting the encoding rule in (|32] | as 



i=0 



and comparing it with the decoder's estimation rule in (|34] | 
we have D„ = A-"X„. Hence, Kjj^ = A-^KniA'y'" with 
diagonal elements E(|D„(j)p) = Pj"^"^ Kn{j, j) and (gO)! can 
be written as 



N 



pin) <J2KnU,j) ■ 22"(«^-l°gft). 



(41) 



But by Lemma |6] \im sup^_^^ Kn{j, j) < oo. Therefore, 

Pi"^ — )• as n oo. 

pair (A, b) is said to be detectable if there exists a column vector c such 
that all the eigenvalues of A — be' lie inside the unit circle. For a diagonal 
matrix A = diag(Ai, . . . , Ajv), the pair (A, 1) is detectable if and only if 
all the unstable eigenvalues Xj, i.e., the ones on or outside the unit-circle, are 
distinct jTS] Appendix C]. 



Finally, by Lemma |6] and the Cesaro mean lemma ||20l , the 
asymptotic power of sender j satisfies 

n n 

lim - J2 E(^^) = lini - J2^K.)» = 



Hence, Kramer's code satisfies the power constraints 
Pi, ... , Pm for sufficiently large n, provided that 



K*^<Pj, j = l,...,iV. 



(42) 



(36) This completes the proof of Theorem |2l 



C. Achievability Proof of Theorem Q] 
Fix any /3 > 1 such that 

NlogP <Cl{P,N), 



and choose 



/3, 



(43) 

(44a) 
(44b) 



for j — 1, . . . , N. Under this choice of parameters, by Theo- 
rem |2] Kramer's code achieves any sum-rate R < N\og(3 < 
Cl{P,N) provided that (gSJi holds. To show (|42| we use the 
following lemma (see Appendix |E] for a proof). 

Lemma 7: When A is defined through ( |33] ) and (l44l) . then 
the unique positive-definite solution K* to the DARE OTT l is 
circulant with all real eigenvalues satisfying Xj — Aj_i//3^, 
j ~ 2, . . . , N, and with the largest eigenvalue Ai satisfying 



d2N 



1 + Xi{n-j^ 



1 + A^Ai = 

-^1 \ n2{N-l) 



(45) 
(46) 



Now by the lemma and the standing assumption ( |43l ) on /?, 
we have 

ilog(l + 7VAi) < Cl(A^,F) 



-log(l + iVP0(iV,P)) 



Thus, 



Ai < P(j){N,P). 
On the other hand, from (l45l l and (|46] | we have 



(47) 



(1 + A^Ai) 



N-l 



1 + Ai iV 



A. 



N 



Hence, by the definition of the function 0(iV, P) in 



(48) 



Combining ^} and (ggll, we obtain K*-(t){N,K*-) < 
P(t){N,P). Finally, by the monotonicity of (j){N, ■) (see Ap- 
pendix |A]i, we conclude that K*^ < P, j ~ 1, . . . , N, which 
completes the achievabiUty proof of Theorem [T| 
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V. Discussion 

In this paper, we established the linear-feedback sum- 
capacity Cl {N, P) for symmetric power constraints P. Below, 
we discuss the complications in extending our proof technique 
to establish the linear-feedback sum-capacity under asymmet- 
ric power constraints or the sum-capacity C{N, P). 

A. General Power Constraints 

The main difficulty in generalizing our proof to asymmetric 
power constraints (Pi, . . . ,Pn) lies in extending Lemma 4. 
The proof of Lemma |4] heavily relies on the fact that covari- 
ance matrices of the form ( l27l ) are optimal for the optimization 
problem in ( |23] |. This allows us to reduce the optimization 
problem ( l23b over covariance matrices to the much simpler 
optimization problem in ( l24b over only two variables x and 
(f). However, covariance matrices of the form ( |27l ) are not 
necessarily optimal for the equivalent optimization problem 
under asymmetric power constraints. 

B. Sum-Capacity 

It is commonly believed that under symmetric power con- 
straints the linear-feedback sum-capacity generally equals the 
sum-capacity, i.e., C{N,P) — C-l{N,P) for all values of 
P and N (cf. [HI). However, currently a proof is only 
known when the power constraint P is larger than a certain 
threshold — the unique positive solution to (|4|i — that depends 
on N |8|. The main difficulty in establishing this conjecture 
for all values of P > lies in proving that Lemma [T] also 
holds for C{N,P). The rest of the proof remains valid even 
for arbitrary (nonlinear) feedback codes. 

Below, we provide an observation based on the properties of 
Hirschfeld-Gebelein-Renyi maximal correlation |14|, which 
further supports the conjecture that C{N, P) = Cl(A^, P). 

C. Greedy Optimality of Linear-Feedback Codes 
Let 

1 " 

C(")(P) :=max- V/(XH,...,X^,;y,|r^-i), (49) 
n ^ — ' 

i=l 

where the maximum is over the set of arbitrary functions 
{Xji{Vj ,Y^~^)} satisfying the symmetric block power con- 
straint P and Vi , . . . , Vn are independent standard (real) 
Gaussian random variables. As shown in Appendix |G] the 
sum-capacity is upper bounded as 



C(P) < lim sup C(")(P). 



(50) 



We introduce a new notion of conditional maximum corre- 
lation to show that for every n linear functions are greedy 
optimal for the optimization problem defining C*^"' (P) in ( |49] l. 

Recall that the maximal correlation p*(Vi, V2) between two 
random variables Vi and V2 is defined [T?] as 

P*iVi,V2) := sup £(51(^^1)32(^2)) (51) 

where the supremum is over all functions gi{vi) and 52 (''^2) 
such that E(gi(Fi)) = £(32(1^2)) = and E{gl{Vi)) = 



E(g|(V2)) = 1. We extend this notion of maximal correlation 
to a conditional one. The conditional maximal correlation 
between Vi and V2 given another random variable (or vector) 
Y is defined as 

p*{y^,V2\Y) := sup E(gi(Fi,r)52(l^2,r)) (52) 

91,92 

where the supremum is over all functions gi [vi , y) and 
g2{v2.v) such that £{gi{V^,Y)\Y) = £(g2{V2,Y)\Y) = 
and £{gl{Vi,Y)\Y) = £{gl{V2,Y)\Y) = 1 almost surely. 
The assumption that gi(Vi,F) and g2{V2,Y) are orthogonal 
to Y is crucial; otherwise, both could be chosen as functions 
only of Y and p*{Vi, V2\Y) = 1 trivially. 

Let p(Vi, V2) denote the correlation between Vi and V2. We 
define the (expected) conditional correlation between Vi and 
V2 given Y as 

p{V^,V2\Y) := j p{V^,V2\Y = y)dF{y), 

where p(Vi,V2|l^ = y) denotes the correlation between 
Vi and V2 conditioned on Y — y. It can be shown (see 
Appendix 101 that if (Vi, V2,Y) is jointly Gaussian, then 

P*{Vi,V2\Y)^p{Vi,V2\Y) 

and linear functions gi and 52 of the form 

..(^.r).^=a^, ,.1,2, (53) 



E{{V,-E{V,\Y)y 

attain p*{Vi,V2\Y). 

Back to our discussion on C'^"^(P), consider the case 
TV = 2 for simpHcity. Then, C^") (P) is upper bounded (see 
Appendix IHl) by 

1 " 

max max — log ( 1 + P^ + P2i 
{P i} 2n V 

+2^PuP2^p{Xu.X2^)] (54) 



where Xji = Xji 



j = 1,2, i = 

the inner maximum is over the set {Xji(Vj, F*^^)} satis- 
fying E(X|j) — Pji; and the outer maximum is over the 
set {Pji} satisfying Y!h=i ^i* — Suppose that linear 

functions Xji = Lji{Vj,Y'^~^) are used up to time i — 1 
and therefore (Vi, V2, 1^*~^) is jointly Gaussian. By definition, 
p{Xu,X2^) < p*iVi,V2\Y'-'^), which by Appendix E equals 
p{Vi, V2|F*~^) and is attained by linear functions Ln and i2i- 
In this sense, choosing Xji linear is greedy optimal for the 
inner maximization in ( l54l i. Note that when (Vi, V2, 1^*^^) is 
jointly Gaussian, then a linear choice of Xu and X2i implies 
that also {Vi,V2,Y^) is jointly Gaussian. This observation, 
which can be easily extended to any number of senders N, 
further corroborates the conjecture that C-l{N, P) — C{N, P) 
for all symmetric power constraints P > 0. 

Incidentally, global optimality of linear-feedback codes of 
the form Xji — LjiiVj ,Y^^^) would also imply that the 
performance of Kramer's code, which uses complex signaling 
(fc = 2), can be achieved by real signaling. In this case, 
the optimal real signaling would involve nonstationary or 
cyclostationary operations, because a stationary extension of 
Ozarow's scheme to > 3 senders is strictly suboptimal ll2ll . 
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Appendix A 
Properties of 0(iV, P) 

We fix the integer N > 2 and prove that for P > the 
solution ^(iV, P) to (|3]l is unique and increasing in P. Note 
that the identity in Q is equivalent to 



(55) 



where Ci{P,(j>) and C2{Pt4>) are defined in (|26] |. We prove 
the uniqueness of (t){N, P) by showing that /(P, 1) > 0, 
/(P,7V) < 0, and df{P,4>)/d4> < for 1 < < TV. The 
fact that f{P,N) < is immediate. For /(P, 1) > 0, note 
that (1 - l/TV)'' > 1 - A:/iV for iV > 1, or equivalently, 



N 



Thus, 



N 



fc=l 



{N -1)'' > 



N -1 



l<k< N -1. 



N-l 



k=l 



N~ 1 



(56) 



which implies that (A^ - 1)C2(P, 1) > (A^ - l)Ci(P, 1) and 
thus that /(P, 1) > 0. The condition df{P,(i))/dcj) < is 
equivalent to 



N-2<j> 



N-l 



1 + P(I){N - 0) 1 + NP(I) 



< 0. 



(57) 



Rearranging terms in ( |57] i we have 1 + AfP(/!) — (20 + P0^ + 
NPcf)^) < which holds for all > 1. This completes the 
proof of the uniqueness. 

We next prove the monotonicity of (j){N, P) in P. By (O, 
we have 



l + NP<j) 



{i + NP(i>y/^, 



1 + P(j){N - (f) 
or equivalently, 

P(t>{N -4>)^{\ + NP(t>)^^-^^/'^ - 1. 

Moreover, since 1 + P0 > (1 + NPcfyf/^ for A^ > 1, 

1 + NP4> - (1 + NP(t>Y/^ > P(j){N - 1). 

Multiplying (|58] l by ( |59] l and considering (|60] l, we obtain 

1 + NP(j) 
^^-^)- l + P0(Ar-0) >^-^- 

From (|6TI ). it is straightforward to verify that 



(58) 



(59) 



(60) 



a/ 

dP 



Finally, by differentiating 

dP 



> 0. 

P,ct>(N,P) 

we have 



5/ 
5P 



5/ 



(62) 



(63) 



p,<i>{N,p) 



Combining (|62]i, (|63j, and the fact that df/d4> < (shown 
above in we conclude that d<f)/dP > for (P, 0(Af, P)). 



Appendix B 
Concavity of fi{K) and f2{K) 

Our proof is based on the following general lemma. 

Lemma 8: Let (U, V) be a Gaussian random vector with 
covariance matrix AEA' + BB'. Let /(S) /i(U|V). Then, 
/(E) is concave in E )^ 0. 

Proof: Fix A and P. Let Ei, E2, and A e [0, 1] be given, 
and E := AEi + (1 - A)E2. For g = 1,2, let {Uq,Yg) - 
N(0, AT.gA' + BB') and (U, V) - N(0, AY.A' + BB'), and 
let Q be a binary random variable with P{Q = 1} = A = 
1 - P{Q = 2}. Assume that (Ui, Vi), (U2, V2), and Q are 
independent. Then, 

A/(Ei) + (1 - A)/(E2) = MUqIVq, g) 

< /^(UqIVq) 

< h{\5\Y) 

where the last inequality follows by the conditional maximum 
entropy theorem |5, Lemma 1] and the fact that (Uq,Vq) 
has the covariance matrix AY, A' + BB' . ■ 

Now let X{S) - N(0, A') and Y = ^2^=1 + Z, where 
Z - N(0, 1) is independent of X{S). Then, 



h{K) = h{Y), 
f2{K) ^ 



N- 



N 



and the concavity of /i and /2 in K follows immediately from 
Lemma H] 

Appendix C 
Concavity of 5(7, x, (f) in </> 

Comparing the definitions of /i (AT) and /2 (AT) in ( fTSl l with 
the definitions of Ci(x, (/i) and 6*2(2;, 0) in (|26] |. respectively, 
we see that when A' has the symmetric form in (|27] | with 
p = |Et' then /i(if) = Ci(x,0) and h{K) = C2(x,0) = 
f2{K). We prove in the following that for every 7 > the 
function (1 — ^)fi{K) +jf2{K) is concave in K over the set 
of positive semi-definite matrices AT ^ with fixed diagonal 
elements. This implies the concavity of 5(7, x, (f) in (j) for 
fixed 7. 

Let X{S) - N(0, a:) and Y = J^f^i Xj + Z, where Z - 
N(0, 1) is independent of X{S). Then, 



(61) (l-7)/iW+7/2(if) 



(i-7)Mn + 



= {l-^)h{Y) + 



N 

AT- 1 ^ 

N 

7 



1 ^ 



AT- 



AT- 1 



Af- 



h{Y\X,) 



{h{Y) + hiX,\Y) - h{X,)) 



N 

-Y,HXj\Y)^h{X, 



By Lemma|8]in Appendix iBl h{Y) and h{Xj\Y) are concave 
in K. Since h{Xj) = -i log(27reA'jj) depends only on the 
diagonal elements of AT, the claim follows. 



10 



Appendix D 
Properties of g{j, x, 0*(7, x)) in x 

For simplicity, we do not include 7 explicitly in our nota- 
tion: g{x,(t)) := g{'y,x,(p) and (/)* (x) := (p*{'^,x). We first 
show that g{x,(j)*{x)) is monotonically nondecreasing in x. 

= 0, we obtain 



Since 4'*{x) satisfies (|28] | and '^^g^''^-' 
dg{x,(t)*{x)) 



dx 
dgjx, (f>) 
dx 

dg{x, (j)) 



dg{x, 4>) d(j) 
dd> dx 



Xjtf)* {x) 



dx 



..4>*{x) 



-fN(t>{N - (t>) 



2{l + Nx(j)) 2{N ~ + x(j){N 

7V(7-l)(rW)^ 
2(1 + Nx(l)*{x)){N ~ 2(j)*{x)) 

>0, 



x.(p* (x) 



(64) 
(65) 



where (|64l l follows by ( |28T l and ( |65T l follows since (7 — 1) and 
{N-2(j)*{x)) have the same sign (see (|28]l). Thus, g(a;, (/)*(a;)) 
is nondecreasing in x. 

We now show that g{x, cj)* (x)) is concave in x. We first note 
that for < 7 < 1 the function g{x, 0) = (1 - 7)6*1 (x, + 
7(72(0;, (/)) is concave in {x, (p) because for symmetric matrices 
K of the form in ^ with p = ^ both Ci{x, 0) = /i(i^) 
and C2(a;, 0) = J2{K) are concave in K (see Appendix iBli. 
Thus, for any v G [0, 1], xi,X2 > 0, and x = vxi + (1 — v)x2, 

vg{xi,(j)*{xi)) + (1 - v)g{x2,(l)*{x2)) 

<g{x,v^*(x^) + {l-v)^*{x2)) (66) 
<g{x,(p*{x)), (67) 

where (|66] | follows by the concavity of g{x, (p) and (|67] | fol- 
lows by the definition of (/>*(x). This establishes the concavity 
of (f>{x,(l)*{x)) for < 7 < 1. 

To prove the concavity for 7 > 1, we show that the second 
derivative d'^g{x,(l)*{x))/dx'^ is negative. Define 



h{x,(j)) 



{1 + Nx(f>){N -2(j>)' 



(68) 



Then, by (|64] |. 

d^g{xA*{x)) 
d'^x 

dh{x, 0) 



dx 



iV(7-l) 

x^(p* (x) 



dh{x, (f>) 



x^cj}* (x) 



d(t>*{x) 
dx 



{1 + Nx(l))^{N -2(j}) 



(j>{N^x(f) + 2{N - (j>)) 



X,(p* {x) 



d(j)*{x) 
dx 



{l + Nx4)Y{N -2(t)y 
^^(iV^x.^ + 2{N - 4))) - N(t?{N - 2(1,) 



{1 + NxcpYiN -2(j)Y 



Xjtf)* {x) 



Since the denominator and 4)*{x) are positive, the following 
inequality concludes the proof of concavity for 7 > 1: 



d(t)*(x) ^ N(j)*{xY{N -2(j)*[x)) 



(69) 



dx " N'^x(l)*{x) + 2{N - (j)*{x)y 
We now establish ( |69] l. Rearranging terms in ( |28] l, we obtain 



that (jf [x) is the solution to the quadratic equation 

acj)^ + b<p + c = 0, 



(70) 



where a = (A^ + 7- l+ ^N)x, h = -N{N + 7 - 1)2; + 27, 
and c = — (A^ + 7 — 1). Since ac < 0, there is a unique positive 
solution 4'*{x) = {—b + Vb'^ — 4ac)/2a. Taking the derivative 
of (ITOI i with respect to x, we find 

drjx) _ -iq^*ix))^ia'(j)*{x)+b') 
dx a{(j)*{x))'^ — c 

^ Nirix))HN-arix)) 
aNx{(l)*[x)Y + N ' 

where a' = iV + 7 - 1 + 7iV and 6' = -N{N + 7 - 1) 
are derivatives of a and b with respect to x, respectively, and 
a:= l+7iV/(A^+7-l) S (2, Note by simple algebra 

that a{b' /a'Y — b{b' /a')+c > 0. Because 0*(a;) is the unique 
positive solution to (|70] i with a > 0, we have (t)*{x) < ~b' /a', 
or equivalently, a'(j)*{x) + b' < for every a; > 0. Hence, 
(j>*{x) is strictly increasing in a; > and 0*(x) > 0*(O) — 
(iV + 7 - l)/27. Therefore, 



N - {a~2)(f)*{x) a(l)*{x) 

N ^ TV ' 

On the other hand, since a > 2 and for q,s > 

P ^ r p p + r 
- < - it and only it - < , 

q s q q + s 



(72) 



(73) 



we have 



N - acj)* {x) -{a- 2)0* (x) 



N-24)*{x) ' N 
which, combined with (l72b . implies 

N-a(t>*{x) a(t)*{x) Nx(t)*{x) + l 



< 



N-24>*{x) N Nx4>*{x) + 1' 
Applying (l73T l to (l74b once again, we obtain 



(74) 



N-a(l)*{x) 



< 



aNx{(l)*{x)f + N 



N - 2(j)* (x) N^x(j)* (x) + 2{N - (j)* {x)) ' 
which, combined with (ItTI i. establishes 



Appendix E 
Proof of Lemma|7] 

We first show that the circulant matrix K with all real 
eigenvalues satisfying Ai = \i-i/0^ for i = 2,...,N, and 
with Ai satisfying (|45]l is a solution to the DARE dSB- We 
then show that this also implies ( l46l ). 

Recall that every circulant matrix can be written as 
QAQ', where Q is the TV-point discrete Fourier trans- 
form (DFT) matrix with Qjk = -l=e-^''^^^-^'>^''-^^/^ 
and A = diag(Ai, . . . , Aa?) is a diagonal matrix. We 
can therefore write K — QAQ', and rewrite the DARE 
(EB as A = {Q'AQ)A{Q'AQy - ((Q'AQ)A(Q'1))(1 + 
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l'QAQ'l)-i((Q'AO)A(0'l))'. By our choice of ^ in (l33]i 
and (l44b . and since Q is the TV-point DFT matrix, 

/ A2 




{Q'AQ)A{Q'AQy = /J^ 



(Q'AQ)A(Q'l) = 




A3 



\ 





V 







Ai ; 



and the DARE in dSTI ) can be expressed in terms of diagonal 
matrices only. Thus, in this case the DARE is equivalent to a 
set of equations, where the first A^ — 1 equations are 



Aj - /^^Aj+i, 
and the A^-th equation is 



j = l,...,Af-l, 



A 



N 



/3'Ai 



1 + A^Ai ■ 
for i 



(75) 



(76) 



By (|45]l and since A^ = Ai_i//3^ for i = 2, . . . , A^, we 
conclude that K satisfies ( |75l l and ( |76b , and hence is a solution 
to the DARE 

To prove ( |46] ), we notice that by the DARE dsTl i the diagonal 
entries of K must satisfy 



2 (EaLi -^jfc)^ 

(i + i'a:i) ' 



(77) 



Also, since Q is the A^-point DFT matrix, Ai — X^ILi ^ifc' 



and since K is circulant, X^feLi -^jfe = 'l2k=i ^ife f*^'" 
j = 1,...,A^. Thus, I'Kl — NXi. Combining these two 
observations with ( iTTl i. we obtain 

13^^(1 + NXi)/{l + Xi{N- Ai/if,,)), 

which, combined with i45[ . yields ( |46] | (with K replaced by 

K*). 

Appendix F 
Conditional Maximal Correlation 

Let (Vi, V2, Y) be jointly Gaussian. Then, the pair (Vi, V2) 
is jointly Gaussian also when conditioned on {Y = y}, and 
the conditional correlation p{Vi,V2\Y ~ y) does not depend 
on y and 

piVi,V2\Y^y)^piVi,V2\Y), (78) 

where we recall that p{Vi,V2\Y) ^ J p{Vi, V2\Y = y) dF{y). 
Moreover, by the maximal correlation property of jointly 
Gaussian random variables ll22l . for every y. 



= /f sup E {gi{VMV2)\Y = y)] dFyiy) 

J V 91,32 / 

= p{Vi,V2\Y) (80) 

where the equality in (ISOl l follows by (fTSl l and (|79] l, and 
because gi and 52 are zero-mean for each y. 

Verifying that the linear functions gi and 92 in ( l53T l satisfy 
E(gi|r) = E(g2|r) = 0, E{gl\Y) - E(g||y) = 1, and 
E{gi{Vi,Y)g2{V2,Y)) = p{VuV2\Y) concludes the proof. 
Note that the proof remains valid also when F is a Gaussian 
vector (instead of a scalar). 

Appendix G 
Upper Bound on C(P) 

By the standard arguments, we have 

1 " 

C{P) < limsupmax-^/(XH,...,Xw,;K,|y^-i), (81) 

i— 1 

where the maximum is over the set of arbitrary functions 
{Xji{Mj,Y'^^^)}. Define now for each n an A^-tuple of 
independent auxiliary random variables Ui, . . . ,Un, where Uj 
is uniformly distributed over [0, 1]. Also, let 

Vn ■= $" ' 

where $ denotes the cumulative distribution function of a stan- 
dard Gaussian random variable. Since {Mj — l + Uj)/2^'''^^-'^ is 
uniformly distributed over [0, 1], Vj ~ N(0, 1). Furthermore, 
by the strict monotonicity of $, it is possible to reconstruct 
Mj from Vj. Hence, the set of feasible functions in dsTl i 
can only increase if we consider {Xji{Vj,Y^^^)} instead of 
{Xji{Mj, Y^^^)}. This establishes the upper bound in 



Appendix H 
Upper Bound on C(")(P) 

Let X,, X,, - E(X,,|y'-i) and Y, := Xu + X2^ + 
Zi. It is not hard to see that E(X|J < E(X|j) < Pji and 
E{Xj^\Y'-^) = for i = 1, . . . , n and j = 1, 2. To establish 
the upper bound ( |54| |. consider 

n 

- V/(XH,X2.;y.|r^-i) 

n ^-^ 

1=1 

1 " 

= -Y I{Xu,X2^,%\Y'-^) (82) 
1=1 
1 " 

<-Y^{Xu,X2^,%) (83) 



1 

%yxvE{g^{V^)g2{y2)\Y ^y)= p(yx,V2\Y = y) (79) < — ^ log (l + Ph + P2. + 2/p^p(1h, 



when the supremum on the RHS is over all functions g\{v\) 
and g2iv2) (implicitly dependent on y) that are of zero mean 
and unit variance with respect to the conditional distribution 
of {Vi, V2) given {Y = y}. Hence, 

p*{Vi,V2\Y)^ sup / E{g^{Vi,y)g2{V2,y)\Y ^y)dFY{y) 

91,92 J 



(84) 

where the equality in ( |82] | holds because E{Xji\Y'^^^) is a 
function of y^^; the inequality in dSSl l follows since Yi — > 
(Xii, ^2^) — >■ form a Markov chain; and the inequality 

in ( |84| | follows by the maximum entropy theorem 1.16,1 and the 
fact that E(12) < Pj,. 
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